educ639_lu_learning_analyticsfandomcom-20200213-history
Learning Analytics Wiki
Learning Analytics in Online Assessments Group 1 Tara Apgar, Arnold Corneal, Angela Miller, Summer Soltis Liberty University EDUC 639-D01 Dr. Daniel Baer April 28, 2019 Abstract Higher education continues to promote innovation and discoveries in the field through learning analytics (LA). Educators are immediately impressed by data and LA and quite often, become sidetracked or biased as to some of the implications or shortfalls of LA. LA shows excellent promise as a field of study, but higher education practitioners need to become more familiar with the issues related to the use of LA. A limited number of studies have examined and assessed previous studies to provide a useful overview of LA issues and the way it impacts higher education. This literature review attempts to give an overview of benefits, risks, and challenges of using LA in an online environment. The results of this review show that LA uses various methods including data mining, digital tracking, and visualization. The review initially started with over fifty articles that were culled down to fifteen after careful consideration. Furthermore, the review revealed that data analysis was used to include prediction, clustering, relationship mining, discovery with models, and separation of data for human judgment to analyze data. There are challenges to overcome so that LA can be applied more effectively to improve teaching and learning in higher education. Keywords: learning analytics and methods, online assessments, and learning analytics, student evaluation, learning analytics and education, and learning analytics. Introduction Online education has become a big part of the education system today, and as education grows, and changes with the passage of time, the online component is driving change in a new direction. Education is being offered via an online format, but the question remains if this new format is still providing for genuine learning. Through this new platform, new ways of self-reflection are available. The introduction of learning analytics goes back to when “data-driven decision making became popular which was in the 1980s and 1990s” (Picciano, 2012). This was the time that online learning and blended learning starting taking off. It was not only in colleges, but it went into the workforce as well. It has changed our society forever. Analytics have existed in many forms throughout the years through various forms of analysis; however, now a new set of data is available for the analyzing. Learning analytics (LA) will allow for the identification regarding the impact of the latest assessment strategies as well as their reliability and validity. This will allow for the useful measurement of cognitive skills, social-emotional development, and deeper learning. The objective is to determine whether these assessments will provide both students and instructors with definitive data that would guide and influence online instructional design modules. The primary aim of the literature review is to draw reasonable conclusions as to the benefits of learning analytics (LA) as a resource that can be used to gather direct evidence of student learner gains. Furthermore, the review will be used to determine whether LA can be used to guide teacher actions and inform curriculum development. Literature Review Explanation of Learning Analytics Picciano (2012) spoke of the beginning of big data and data analysis back in the 1960s to manage finance and monitor student progress. In the 1970s, other applications began to be used, and data management and analysis spilled over into new areas until it reached a point the more similarly resembles the LA of today in the 1990s. The 90s was the beginning of the migration of personal data to a digital format and entered in a new era of distance communication and information sharing. Today, everything is integrated into social networks, and learning is done online as well as blended learning formats. This has led to a push to use data to make informed decisions that are driven by data which ushers is LA (Picciano, 2012). The advancement of technology has provided the opportunity to track and store student learning activities as effective big data sets within online environments. Big data is what it purports to be, a significant quantity of data that must be stored and can come from a variety of sources (Avella, Kebritchi, Nunn, & Kanai, 2016). LA seeks to improve learning by analyzing data that has been collected and stored from online sources. These analytics are then used to provide the basis for informed decisions about future direction. Even though LA is still very much emerging in the field of education, many in the field of online learning predict that within the next few years learning analytics will be widely used in online/distance education. The author asserted that LA will eventually be used to help schools and institutions recognize patterns in student behaviors and that this information can help enact changes that will raise the student retention rate (Avella et al., 2016). Learning analytics uses predictive models that provide tangible information that may be easily understood and applied realistically. It is a multi-faceted approach that revolves around data processing, technology-learning enhancement, educational data mining, and visualization (Rienties, Boroowa, Cross, Kubiak, Mayles, & Murphy, 2016). The objective of LA is to develop educational opportunities to the individual learner’s specific needs. It can lend itself to adopting a scientific approach to engage students in a personal way that allows the educational experience to be aligned with all their learning attributes, shortcomings and characteristics. It is also able to provide ongoing formative feedback that can be used for adjustments to be made as instruction continues. Due to the digital nature of this feedback, it can be made available in near real time, thus potentially providing valuable information at the moment that it is beneficial (Picciano, 2012). Data Mining Educational data mining focuses on developing and implementing methods to promote discoveries from data in educational environments. Vahdat, Ghio, Oneto, Anguita, Funk, & Rauterberg (2015) defined data mining as data analysis techniques as a means to extract hidden knowledge consisting of specific tasks, using a pattern discovery as well as predictive modeling. The definition of educational data mining is one that uses data mining algorithms as a strategy to solve educational problems. Academic analytics refers to the application of the principles and tools of business intelligence in academia with the specific goal of improving decision-making and performance within educational institutions (Sayed & Jradi, 2014). Data mining can take many forms, can benefit one group more than another, and be useful in a limited number of circumstances (Nyland, Davies, Chapman, & Allen, 2017). This is because data mining can be beneficial at three different levels. According to Nyland et al. (2017), these levels are; first, the broadest is the institutional level, such as courses completed. The second is an assessment level where much research has been done especially in regards to high stakes testing. However, the authors present the idea that a third more specific level can be even more useful, and this is the “transaction-level data” relating to the student level, whereas they progress through instruction, information is available to guide instructors to offer specific tutoring to correct in the moment errors (p. 203). They cautioned that data mining might not be beneficial in all circumstances (Nyland et al., 2017). Image by mcmurryjulie from Pixabay Digital Tracking The foundations of digital tracking reside with information and how it is applied in the process of analysis. Its main objective is to determine the most productive ways to provide new learning modalities and opportunities. The speed of the application of technology has researchers already thinking about its impact into the second decade of the 21st Century (Kovanovic, Gasevic, Joksimovic, Hatala & Adesope, 2015). Kovanovic et al. (2015) asserted that data tracking is automatically collected by the learning management system and is useful for tracking the number of interactions with the material as well as the time that the interactions occurred. This may or may not be useful information depending on what the intended goals are as well as how the students actually use the system. They discussed that this data is relatively easy to interpret but ultimately may be less useful (Kovanovic et al., 2015). The component of monitoring depends on the tracking efficiencies of big data. Current trend tracking indicators depend on the learning management system used by the institution. Visualization Avella et al. (2016) found that visual data analysis can be complicated due to the sheer size of the data to be displayed. However, they asserted that with the proper tools, vast datasets could be reduced into manageable groupings and thus displayed in beneficial ways to show the potential trends as well as patterns. Piety, Hickey, & Bishop (2014) discuss the value of visualizations as they are often used as “dashboards” with quick visuals to aid the reader in understanding the data (p. 6). They did caution that these visualizations are somewhat subject to interpretation and can, therefore, be affected by the cultural bent of the one observing the display. Benefits Learning Analytics and by extension computer-aided methodologies, seemingly here to stay, do offer a lot of advantages in terms of educational progress. Improved student learning outcomes, curriculum development, personalized learning advancements, better communication strategies, as well as advancements in testing policies all are possible gains to be seen from the application of LA (Wang, 2017; Wise, A. F., Zhao, Y., & Hausknecht, S. , 2014; Timmis, Broadfoot, Sutherland, & Oldfield, 2015; Scheffel, Drachsler, Stoyanov, & Specht, 2014). It could be argued that as learning analytics are applied more at the micro-level, gains in learner advancement will continue as learners and instructors have more access to meaningful, purposeful data. It is at the micro-level that behavior modifications are enhanced. While, at the macro-level, there is a potential for educational policy reform. Macro-Level The use of computer-based assessment for learning, (CBAfL), leads to almost limitless new forms of evaluation. This allows for improved forms of personalized learning through more formative structured evaluations (Shute & Rahimi, 2017). The addition of the beefed-up role of formative assessment begins to take on a more significant role in moving students through the material when the data is reliable and testing policies and strategies stable (Shute & Rahimi, 2017; Wise, Zhao, & Hauskenecht, 2014; Timmis et al., 2015). There has been substantial innovation that has occurred due to the advent of LA. Researchers have presented clear evidence that suggests, as a result of LA, there are now new forms of representing knowledge and skills, and this may lead to broad, sweeping changes in assessment. Shute & Rahimi (2017) suggested that the primary focus of assessment should be limited to students, but it must do much more, and should be done for the benefit of students. The objective must be to guide and enhance their learning. Assessment must have specific components, such as interpreting and acting on information about learners’ understanding and/or performance. This, however, must be done in correlation to pre-set educational goals. It is the hope that as macro-level data is gathered from LA, new innovative policies will arise for addressing those abilities. Micro-Level There have been significant benefits that have been derived from the availability and analysis of Learning Analytics, making it now possible to develop a view of student progress (Avella et al., 2016). STARBURST (Wise et al., 2014) was a developed technology to provide a visual representation of data for discussion board participation. It was very beneficial for students to see if they were reading or scanning posts, where they were interacting, and how ideas were connected. As shown with STARBURST, when student and teachers have access to appropriate data, they can begin to change their learning and teaching. Giving students access to their data and class averages allows them to set goals for improvement that may not be measured by an exam but improve learning quality. Teachers can see where they need to change or modify courses or content in real time as well. Learning analytics should not be limited to macro-data but should include micro-data that is used at the course level. This information is valuable for changing behavior mid-course rather than after the fact and as Picciano (2012) mentioned, greater individualization is available to the student. As technology increased the ability of students and instructors to change behavior mid-course and has added to the different types of data that can be extracted, personalized learning is becoming stronger and stronger. Furthermore, these new technologies allow for the identification of specific areas of strength and weakness allowing for genuinely personalized learning as well as personalized assessment. Timmis et al. (2015) looked at the overall impact of the new technologies and determined that they have dramatically changed the environments and processes by which students learn. It is also apparent that how students and teachers communicate have been replaced both with each other and peers. Kovanovic et al. (2015) postulated modern technology provided ample opportunity for effective social interaction with both peers and instructors. Innovative ways of interacting along with high impact data have a strong influence on how instructors communicate feedback as well as what they communicate as feedback. Exposure to all of this data, however, does not lead to automatic growth. Shute and Rahimi (2017) showed that instructors need to be careful with the elaborate feedback given to students. If the detail was too overwhelming, the feedback had less benefit than simple verification feedback (right or wrong). However, when feedback included small amounts of targeted data, the effects were far more significant. Therefore the application of such large amounts of data produced by LA must be carefully targeted for the different initial users of researchers, instructors, and students. Risks and Challenges While there is no doubt that learning analytics present clear and positive opportunities to the field of education, it is imperative that the full impact of the field be reviewed. There is an inherent risk when using data obtained for learning analytics purposes. The amount of data, technological dependencies, high stakes testing, social justice concerns, and ethical implications all have been addressed in the literature as presenting issues to the field of learning analytics. Each offers its own unique set of considerations. Data Volume The issues to data volume are twofold: the sheer volume of data and getting the analytics into the right hands in a timely enough manner to be of any real benefit to the learner (Tempelaar, Rienties, & Giesbers, 2015). As the technology has improved, so has the size of the data set provided to researchers connected to LA. While the quality of this data is also to be questioned, at times, the quantity can be so overwhelming that valuable information is lost. Teachers and researchers seem to lose focus and do not dedicate sufficient attention to the actual benefits, limitations, or implications of LA in education. Tempelaar et al. (2015) posited that it is essential to use previous studies of LA to weed out the data that is useless for prediction such as program clicks. This would reduce the amount of data collected, which would, in turn, refine the data collected to areas or issues that would indeed be beneficial. However, Picciano (2012) was concerned for those that lack the training to manage the quantity of data currently collected. The lack of trained analysts could mean data ends in the wrong hands due to outsourcing the analytics (Picciano, 2012). All of this refers back to macro-data. However, it is at the micro-data level that teachers and students operate to effect change in learning behaviors and outcomes. The Templelaar et al. (2015) study, focused on the issues of timely feedback learned through LA. If the LA is performed too late in the learning process, then that feedback will not be available in time to allow the learner to enact real change. The same effect would happen with any LA that requires lengthy collection and analytics time thus producing the same result of delayed feedback to the learner (Tempelaar et al., 2015). Therefore, the use of dashboards with quick views of scores, progress, and other micro-data visualizations need to be presented to both instructors and students. The issues to data volume are twofold: the sheer volume of data and getting the analytics into the right hands in a timely enough manner to be of any real benefit to the learner (Tempelaar, Rienties, & Giesbers, 2015). As the technology has improved, so has the size of the data set provided to researchers connected to LA. While the quality of this data is also to be questioned, at times, the quantity can be so overwhelming that valuable information is lost. Teachers and researchers seem to lose focus and do not dedicate sufficient attention to the real benefits, limitations, or implications of LA in education. Tempelaar et al. (2015) posited that it is essential to use previous studies of LA to weed out the data that is useless for prediction such as program clicks. This would reduce the amount of data collected, which would, in turn, refine the data collected to areas or issues that would indeed be beneficial. However, Picciano (2012) was concerned for those that lack the training to manage the quantity of data currently collected. The lack of trained analysts could mean data ends in the wrong hands due to outsourcing the analytics (Picciano, 2012). All of this refers back to macro-data. However, it is at the micro-data level that teachers and students operate to effect change in learning behaviors and outcomes. The Templelaar et al. (2015) study, focused on the issues of timely feedback learned through LA. If the LA is performed too late in the learning process, then that feedback will not be available in time to allow the learner to enact real change. The same effect would happen with any LA that requires lengthy collection and analytics time thus producing the same result of delayed feedback to the learner (Tempelaar et al., 2015). Therefore, the use of dashboards with quick views of scores, progress, and other micro-data visualizations need to be presented to both instructors and students. Dependency on Technology There is a concern that there may be increasing dependence on technology in future assessment practice. Instructors are suspicious of technology-enhanced assessment (TEA). These suspicions are not unwarranted if the TEA is applied in isolation, without human intervention or interpretation (Timmis et al., 2015). Timmis et al. (2015), further point out that TEA could negatively affect student self-efficacy and learning goals or aspirations. High-Stakes Testing One of the toughest challenges of learning analytics is the use of big data and high-stakes testing. Current assessment policies and practices are called into question for the reliance on these high-stakes test. The idea that the tests are misaligned to the curriculum is not uncommon. It could further be argued that there is an over dependency on the grading without a severe enough concern for the testing methodology and practices. Timmis et al. (2015) suggested that the risk of widespread LA for school improvement and benchmarking for high-stakes testing is “insidious” (p. 12). They also purported that this digital deluge can lead to misinterpretation as well as skewed or misleading data. As the educational community is accepting the changing landscape of knowledge, how high-stakes testing is observed is also changing. However, any adaptions that are being made are sporadic and isolated either with individual use or single organization use (Timmis et al., 2015). Timmis et al. (2015) presented the argument that “ a recognition that conventional methods of assessment, tried and tested as they have been over more than a century, are increasingly unfit for purpose” (p. 4). They further pointed out that the technology that helps support learning analytics is still typically using multiple choice questions and relying mostly on the transmission method for learning. Furthermore, as of today’s students and education shift, there is more focus on the ever-increasing global connection and global citizenship. Collaboration is highly valued yet very few programs are available to mine data from collaboration opportunities. Wise et al., (2014) did create STARBURST to mine and monitor collaboration data via both embedded and extracted data, but this is a singular model not employed in many institutions despite the valuable information that is obtained from there. Digital Divide The reliance on digital technologies to assist LA also emphasizes the digital divide (Timmis et al., 2015). The familiarity, or lack of familiarity, with the technology used for the assessment, can skew the data used in the analytics. Despite data appearing neutral and unbiased, it is not as it is determined to be of importance by biased formers. The tools used for data mining reflect the values, and therefore the biases, of the practices to which they are associated (Timmis et al., 2015). While technology is ever increasingly at our fingertips, the full capabilities of how students are asked to collaborate or present their learning is not equally available. Ethical Concerns There are ethical concerns associated with the ownership of the data mined through LA. With businesses having increased access to the mined data through internet browsing history and the like, the digital footprint of data left behind is growing in value and has no regulations for control. Cope & Kalantzis (2016) pointed out this issue in their work on data tracking. The use of LA for international use is on the rise, and this puts the data at risk (Timmis et al., 2015). Students from whom the data is being garnished, have no control over it or their data and Picciano (2012) was concerned that this data ends up accidentally in the wrong hands. There clearly is a need to make the data more visible and controllable by the learners. Timmis et al. (2015) argued that the digital tools should “be aligned with an explicit set of pedagogical and ethical principles to cover the purpose, access, ownership, and control of data” (p. 16). Sharing data with parties outside of the institution must also be taken into consideration. Furthermore, there is a question of who owns this aggregated data. The vast extent of this data becomes extremely difficult to safeguard. Learning institutes must, therefore, find innovative ways to protect the data of their students. This will become an ever-evolving endeavor (Rienties et al., 2016). While technology creates the data, it is also used to store the data. Increasingly, education is becoming on cloud-based data storage which also creates privacy concerns (Sayed and Jradi, 2014). Image by Peggy und Marco Lachmann-Anke from Pixabay Recommendations It is easy to see the benefits of applying learning analytics to different levels of the educational system. However, based on the inherent risks associated with learning analytics, there are some recommendations for moving forward with the application. High-Stake testing Learning analytics on the macro level is imperative for school improvement. The tying of learning analytics’ data to high-stakes testing, however, is a major concern that needs to be addressed. The validity of this information as it applies to specific learners and specific instructional staff is weak in its current state. Micro-level analytical data is more appropriate for the strings currently tied to these tests. While more and more states are moving away from linking the passing of high stakes testing to graduation requirements and job security, the practice is still common. The macro-level data accumulated from these tests are more appropriate for understanding needed wide-scale educational or institutional change rather than individual learner success. Learning analytics on the macro level is imperative for school improvement. The tying of learning analytics’ data to high-stakes testing, however, is a significant concern that needs to be addressed. The validity of this information as it applies to specific learners and specific instructional staff is weak in its current state. Micro-level analytical data is more appropriate for the strings currently tied to these tests. While more and more states are moving away from linking the passing of high stakes testing to graduation requirements and job security, the practice is still prevalent. The macro-level data accumulated from these tests are more appropriate for understanding needed wide-scale educational or institutional change rather than individual learner success. Testing The advent of CBAfL allows for better assessments and more appropriate data collection for the needs of today’s educational environment. When the modern education system was developed, the rate of knowledge doubled only every century. By WWII, it doubled every 25 years. Today it is doubling every 13 months, and with the advent of the Internet of Things (IoT), it is estimated that average human knowledge will double every 12 hours (Schilling, 2013). This dramatically impacts what and how our students need to learn to succeed. While, basics of reading, writing, and arithmetic certainly need to be firmly learned along with scientific and historical concepts, it is necessary for today’s students to be problem-solvers, innovative thinkers, and collaborators. These are the skills that need to be analyzed, and CBAfL has the ability to do that while extracting appropriate LA, Educational policy and testing design need to come into alignment with those priorities. Personalized Learning The increase in human knowledge and today’s changing educational needs also are driving the need for more personalized learning. The ability of CBAfL to structure formative assessments and provide structured data that when applied into LA, allows for far better personalized learning experiences that can be empirically backed. While education is slowly beginning to address this issue, it is at the micro-level of application that this is seen. CBAfL and LA clearly add to the ability to create such a method of learning, but educational policies are set up against this currently. Clear guidelines and reforms across all levels and areas of education need to be enacted. Picciano (2012) had high hopes for automated forms of tracking that will predict student success and failure to flag those who need help or challenge to provide the individual assistance needed to provide for the best student outcomes possible, while this may seem like a positive, automating such an important part of a student’s future success also requires great caution to protect individual students. Ethical Issues It is imperative that a focus is made on addressing the ethical and social issues brought up in the collection, use, storage, and ownership of data produced with LA. As the world is becoming increasingly more digitally based and the number of individual digital footprints are growing, there needs to be the protection of rights surrounding that data. Education must do its part to address these concerns and determine a baseline protocol for the data accumulated and how it can be applied and stored. Students’ rights to that data and how it is used must also be addressed. Overall, the ethical and moral obligation of protection of privacy and ownership is a paramount concern that needs to be addressed when entering into the use of learning analytics on any scale. Conclusion The review presented sound evidence that LA consistently enhances learning. Data can be derived through various ways and can be quantified sufficiently to allow educators to adjust instruction in a way that specific student learning needs may be addressed. These findings cover multiple content areas. The ability to receive feedback was most beneficial to learning. This study has revealed various methods used by LA to show how big data can benefit education. More importantly, it also presented some of the challenges that many stakeholders may face as they engage in the process of applying LA to their educational platforms. The field of LA will continue to grow, as the need to cater to large student populations also grows. The data associated with this growth will be a valuable resource once analyzed, to guiding instructional pathways in education. As a result of greater accessibility to data, learning analytics will provide a better understanding of the various complexities of learner behavior. Examination of the components of LA revealed both positive and negative attributes that can add significant value to education as well as expose some concerns. In this regard, it is essential to continue further research that covers a broad spectrum and will hopefully uncover greater possibilities for the uses for LA while remaining cautious to the fact that there may be equal challenges to overcome to ensure its long-term value. Learning analytics is dependent on other areas, and a reciprocal dependence is sometimes established to adequately achieve the goal of improving education. LA offers possibilities that are available through the many large datasets in educational institutions. Learning analytics follows a wide spectrum of data collection, data analysis, prediction model development, intervention, and refinement of the predicted model (Rienties et al., 2016). MultiMedia https://youtu.be/CdQkxp06iIQhttps://youtu.be/CdQkxp06iIQ Question 1. In what ways do learning analytics support personalized learning? 2. Does "learning analytics" provide more data that would allow institutions to better understand individual student needs, in comparison to traditional modes of instruction? If so, give us one example? 3. Which one of the following answers best describes the most important benefits of learning analytics as it relates to assessment via online/distance learning. A. It provides personalized learning and monitors student progress 'B. Research indicates that it is much better than traditional instruction. C. It makes grading papers much more manageable. 4. Identify the most significant challenge for learning analytics in education? A. It will be challenging to implement in higher education because many students are not computer literate. B. To adequately develop secure data tracking and data collection methodologies that ensure student privacy. C. To provide virtual classroom enrollment for a large number of students who prefer distance learning. 5. There is evidence that computer-based assessment for learning (CBAfL) in the classroom, consistently enhances learning. One aspect of CBAfL seems to be the most beneficial. Which one is it? A. It is easy to see when students log in and how long they spend on tests B. Feedback provided by data is effective in guiding individualized academic agendas C. It offers a more productive social component that having students attend traditional classes.'' '''References Avella, J. T., Kebritchi, M., Nunn, S. G., & Kanai, T. (2016). Learning analytics methods, benefits, and challenges in higher education: A systematic literature review. The Official Journal of OLC'', 20(2), 13-29. Retrieved from https://olj.onlinelearningconsortium.org/index.php/olj/article/view/790'' Cope, B., & Kalantzis, M. (2016). Big data comes to school: Implications for learning, assessment, and research. AERA Open, 2''(2), 1-19. https://doi.org/https://doi.org/10.1177/2332858416641907 Kovanovic, V., Gasevic, D., Joksimovic, S., Hatala, M., & Adesope, O. (2015). Analytics of communities of inquiry: Effects of learning technology use on cognitive presence in asynchronous online discussion. ''Internet and Higher Education, 27, 74-89. https://doi.org/https://doi.org/10.1016/j.iheduc.2015.06.002 Nyland, R., Davies, R. S., Chapman, J., & Allen, G. (2017). Transaction-level learning analytics in online authentic assessments. Journal of Computing in Higher Education, 29''(2), 201-217.'' doi:http://dx.doi.org.ezproxy.liberty.edu/10.1007/s12528-016-9122-0 Picciano, A. G. (2012). The evolution of big data and learning analytics in American higher education. Journal of Asynchronous Learning Network'', 16(4), 9-12. https://doi.org/10.24059/olj.v16i3.267'' Piety, P. J., Hickey, D. T., & Bishop, M. J. (2014). Educational data sciences – Framing emergent practices for analytics of learning, organizations, and systems. Proceedings of the Fourth International Conference on Learning Analytics And Knowledge'','' 193-202. https://doi.org/10.1145/2567574.2567582 Rienties, B., Boroowa, A., Cross, S., Kubiak, C., Mayles, K., & Murphy, S. (2016). Analytics4action evaluation framework: A review of evidence-based learning analytics interventions at the Open University UK. Journal of Interactive Media in Education'', 1''(2), 1-11. Retrieved from https://files.eric.ed.gov/fulltext/EJ1089327.pdf Sayed, M., & Jradi, F. (2014). Biometrics: Effectiveness and applications within the blended learning environment. Computer Engineering and Intelligent Systems, 5''(5), 1-9. Schilling, D. (2013). Knowledge doubling every 12 months, soon to be every 12 hours. Industry Tap. Retrieved from: http://www.industrytap.com/knowledge-doubling-every-12-months-soon-to-be-every-12-hours/3950 Shute, V. J., & Rahimi, S. (2017). Review of computer based assessment for learning in elementary and secondary education. ''Journal of Computer Assisted Learning, 33(1), 1-19. https://doi.org/https://doi.org/10.1111/jcal.12172 Tempelaar, D.T., Rienties, B., & Giesbers, B. (2015). In search for the most informative data for feedback generation: Learning analytics in a data-rich context. Computers in Human Behavior 47'', 157-167. doi: 10.1016/j.chb.2014.05.038 '' Timmis, S., Broadfoot, P., Sutherland, R., & Oldfield, A. (2015). Rethinking assessment in a digital age: Opportunities, challenges, and risks. British Educational Research Journal'', 42(3), 1-23. https://doi.org/10.1002/berj.3215'' Vahdat, M., Ghio, A., Oneto, L., Anguita, D., Funk, M., & Rauterberg, G. W. M. (2015). Advances in learning analytics and educational data mining. In ESANN 2015 proceedings: European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, Bruges, 22-24 April 215 Leuven: Katholieke Universiteit Leuven. Wang, F. H. (2017). An exploration of online behavior engagement and achievement in flipped classroom supported by learning management system. Computers & Education, 114, 79-91. https://doi.org/https://doi.org/10.1016/j.compedu.2017.06.012 Wise, A. F., Zhao, Y., & Hausknecht, S. N. (2014). Learning analytics for online discussions: Embedded and extracted approaches. Journal of Learning Analytics'', 1''(2), 48-71. https://doi.org/ https://doi.org/10.18608/jla.2014.12.4 __NEWSECTIONLINK__ Category:Browse